For further properties of kronecker products in the context of GP regression see Ch. 5 of Saatçi After applying the associative scan to these elements using the operator defined by Eq. (34) and Eq. The sequential (CPU) algorithm is competitive when 20 spatial points are used. In addition to the wall-time experiments, we also studied the characteristics of the parallel and sequential filtering approaches on GPUs via performance profiling. We now show that we recover Eq. (7) from standard natural gradients from [ The proof follows by induction. This recovers the CVI update in Eq. (43). The first term is a scalar and so does not affect the final form.

We introduce a scalable approach to Gaussian process inference that combines spatio-temporal filtering with natural gradient variational inference, resulting in a non-conjugate GP method for multivariate data that scales linearly with respect to time. Our natural gradient approach enables application of parallel filtering and smoothing, further reducing the temporal span complexity to be logarithmic in the number of time steps. We derive a sparse approximation that constructs a state-space model over a reduced set of spatial inducing points, and show that for separable Markov kernels the full and sparse cases exactly recover the standard variational GP, whilst exhibiting favourable computational properties. To further improve the spatial scaling we propose a mean-field assumption of independence between spatial locations which, when coupled with sparsity and parallelisation, leads to an efficient and accurate method for large spatio-temporal problems.